Skip to content

Report delivery-target attainment and first-pass merge rate in metrics#47

Draft
mberman84 wants to merge 1 commit into
mainfrom
cursor/delivery-health-metrics-cbd8
Draft

Report delivery-target attainment and first-pass merge rate in metrics#47
mberman84 wants to merge 1 commit into
mainfrom
cursor/delivery-health-metrics-cbd8

Conversation

@mberman84

Copy link
Copy Markdown
Contributor

Why

DeployBot's whole job is getting PRs out the door quickly and with high confidence. The metrics command already measured raw timing percentiles, but it never told operators two things they need to trust the pipeline:

  • Are we actually fast enough? The configured speed targets (ready_to_merge_target_minutes, merge_to_live_target_minutes) were only ever applied to live work in status; the historical view ignored them.
  • Are deliveries clean? There was no signal for how often a deploy merged on the first pass versus needing a repair handoff.

What changed

This extends the existing metrics subsystem (no new commands, options, config fields, or MCP tools):

  • summarize_metrics now accepts optional per-stage targets and reports target_seconds, within_target, and within_target_rate for each timed stage. With no targets supplied it behaves exactly as before.
  • delivery_metrics derives targets from the repository's own policy and applies them where the meaning is unambiguous: queue→merge and merge→live against their configured budgets, and request→live against the combined door-to-door budget. The promotion-only sub-stage is left timing-only to avoid false confidence.
  • delivery_metrics also adds a reliability block — first_pass_merges, repaired_merges, and first_pass_rate — counting recent merges that landed without a genuine (non-transitional) repair handoff. Each sample carries a repaired flag.
  • The human-readable metrics output prints the first-pass rate and per-stage target attainment alongside the existing percentiles.
  • docs/reference.md updated to describe the richer output.

All additions are read-only and flow automatically through the delivery_metrics MCP tool via the existing --json passthrough. No safety invariants of the merge/release path are touched.

Testing

  • python3 -m unittest discover -s tests — 248 tests pass (3 new).
  • python3 -m ruff check . — clean.

New tests cover target attainment in summarize_metrics (including the no-targets backward-compatible path) and end-to-end target attainment plus first-pass/repaired classification in delivery_metrics.

Open in Web Open in Cursor 

Co-authored-by: mberman84 <mberman84@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants